Bayesian Nonparametric Analysis of Reversible Markov Chains

We introduce a three-parameter random walk with reinforcement, called the (θ,α,β) scheme, which generalizes the linearly edge reinforced random walk to uncountable spaces. The parameter β smoothly tunes the (θ,α,β) scheme between this edge reinforced random walk and the classical exchangeable two-parameter Hoppe urn scheme, while the parameters α and θ modulate how many states are typically visited. Resorting to de Finetti’s theorem for Markov chains, we use the (θ,α,β) scheme to define a nonparametric prior for Bayesian analysis of reversible Markov chains. The prior is applied in Bayesian nonparametric inference for species sampling problems with data generated from a reversible Markov chain with an unknown transition kernel. As a real example, we analyze data from molecular dynamics simulations of protein folding.